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Abstract 

This paper is devoted to the analysis of a finite horizon discrete-time stochastic optimal 
control problem, in presence of constraints. We study the regularity of the value function which 
comes from the dynamic programming algorithm. We derive accurate estimates of the Lipschitz 
constant of the value function, by means of a regularity result of the multifunction that defines 
the admissible control set. 

In the last section we discuss an application to an optimal asset-allocation problem. 
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This paper is devoted to the analysis of a general Finite Horizon Discrete- Time Stochastic Optimal 
Control Model, with inequality constraints. The aim of this paper is to give a method to estimate the 
Lipschitz constant of the value function obtained via the classical dynamic programming algorithm. 

The regularity of the value function is related to the regularity of the marginal function Q| , |p2| , 
and, as is proved in the regularity of the marginal function is connected to the regularity of the 
multifunction which defines the set of admissible controls. Therefore, our main result concerns the 
Lipschitz regularity, with respect to the Hausdorff metric || , of this multifunction. 
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Set-valued maps are widely used in optimal control, differential games and their applications to 
mathematical economics and finance, see [0, and jlSfl . In many cases these multifunctions are de- 
fined by means of inequality constraints for a set of functions defined over a manifold that represents 
the control space. Unfortunately this manifold is usually non-regular, as in the financial application 
presented in Section ^. Furthermore, the constraint functions may lose the regularity or be depen- 
dent on each other, at some point. 

To overcome this difficulties, we allow for Lipschitz manifolds and Lipschitz constraint functions, 
provided that the set where either the manifold or the constraints are not regular, or the constraints 
are dependent on each other, can be approximated by points where both the manifold and the con- 
straint function are regular and the constraints are independent on each other. 

We use a quantitative formulation of the Implicit Function Theorem that provides an estimate of 
the neighborhoods where the implicit function is defined. 

Our study is carried out in the discrete-time case, because of its high computational relevance, since 
the regularity properties of the value function can be used to derive a-priori error estimates and 
convergence results of numerical schemes. 

The outline of the paper is as follows. Section [l] introduces the general framework of the optimization 
model and recalls the classical discrete-time DP algorithm. Section || provides the definition of the 
Hausdorff metric and some related results, Section [| provides basic notations and definitions, Section 
U establishes the main regularity results about multifunctions and a Lipschitzian estimate for the 
value function related to the dynamic programming algorithm. Finally, Section ^ applies these 
results to an optimal asset- allocation problem with regulatory constraints. 

1 The Dynamic Programming Algorithm 

In this section we present the model which is the subject of our study in next sections. We consider 
the following discrete-time controlled dynamical system: 

%k+i = fk(xk,Uk,Vk), k = 0,...,N—l 

(1.1) 

Xq = X E X , 

where 

fk ■ X k x Mk x y k -> x k+1 , 

(1.2) 

X k , X N c H m , M k c m n , 

for every k = 0, . . . , N— 1. Here Xk is the state space, Uk the control and yk the random disturbance. 
For every fc, we are given the following constraint functions: 

cf,. .. ,c k jk : M k x A k -> H, j k < n, (1.3) 

where Ak is an open subset of IR m , with Xk C A k ■ 

The set of admissible controls at time t k for the point x € Xk is defined as follows: 

U k {x) = {ueM k : <$(u,x) <0, i = l,...,j k }- (1.4) 

We assume that Uk{x) is non empty, for every x G Xk and that the random disturbance yk is a 
measurable function over a Probability Space (fJ,^ 7 , P) with values in a measurable space (3^,£fc)j 
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where Ek is a sigma- field over 34 . 

The disturbance y^ is characterized by a probability law Pk('), which we assume independent of 
(xk,Uk) and of prior disturbances yu-x, ■ ■ ■ , Ho- 

We call an admissible control law, a set <j) — {uo, ■ ■ ■ , un-i} of functions Uk ■ Xk — > Mk such that 

Uk(x) € Uk{x), for every a; G .Xfc. 

We denote by U the set of admissible control laws. 

Given an initial state x € Xq, the optimization problem consists in finding an admissible control law 
4> € U which maximizes the cost functional 

J^(x)=JE[g(x N )] (1.5) 

where IE denotes the expected value taken over (Q, T, P), and xn is the value at time tjv of the 
state x, according to (1.1). 

The real- valued function g : Xn — > H is called stopping — cost or utility function. 
We want to point out that, eventhough we limit ourselves to this simpler model for the sake of 
simplicity, the discussion in this paper can be easily extended to include running costs or probability 
measures pk depending also on the state and control variable. 



The Classical Dynamic Programming Algorithm consists in solving the problem fll.q ) by means of 
the following sequence of one-step optimization problems: 

Jk{x)= sup JEk[Jk+i(fk(x,u,y k ))], Vie4 0<k<N 
ueu k (x) 

(1.6) 

Jn{x) = g(x), Vie X N . 

The value function at time tk, Jk is defined over the state space X^, and lE^ denotes the expected 
value taken w.r.t. the measure pk over 34- For any given initial state xq G Xq, the value Jo(^o) 
computed by the algorithm equals the optimal cost 

max J^xq) (1.7) 

and the optimal control policy <fi* can be obtained by cf>* — {«q, • • ■ , u *n-i} where u* k , k = 0, . . . , N— 1 
maximizes the right-hand side of (1.6). See [0 for a detailed description of the Dynamic Programming 



Algorithm in the discrete-time case. 



2 The Hausdorff Metric 

We introduce in this section the Hausdorff metric over the space of all compact subsets of a given 
metric space. This metric is used in Section ^ to estimate the Lipschitz constant of the multifunction 
x i-> Uk(x). 

To study the regularity of the value function Jk defined in ( |l.6|) , since the admissible control set 
Uk(x) depends on the state variable, one needs to measure the distance between the admissible 
control sets corresponding to different states of the system. 

We need, therefore, to introduce a distance between sets in order to show some regularity property 
of the function 

A i ► max / 
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with / continuous and A a subset of a separable metric space (M, d). Since we only consider compact 
control sets, we can limit ourselves to introducing the Hausdorff metric on the class of all compact 
subsets of M, denoted by Comp(M). 
For every Ki, K 2 G Comp(M)\{<D}, let 

d H {K 1 ,K 2 ) = inf {e > : K\ C K £ , and K 2 C K{} (2.8) 

where, for any set A C X, A £ = {y : d(y, A) < e} denotes the open ball of radius e around A. 

It is easy to verify that dn is a metric on Comp(M). Furthermore if two points x,y of X are 

regarded as the single point sets {x} and {y} in Comp(M), then 

d H ({x},{y}) = d(x,y). (2.9) 

That is to say, M is isometrically embedded in Comp(M). See || for the properties of the Hausdorff 
distance. 



The following Proposition concerns the Lipschitz regularity of a real valued map defined over the 
space of all compact subsets of a metric space. 

Proposition 2.1 Let f be a Lipschitz continuous function over the space M, and 

f:Camp(M)^M. 

K h+ max/. ^- iU J 

Then f is a Lipschitz continuous map over the metric space (Comp(M),dH) and its Lipschitz con- 
stant equals the Lipschitz constant of f over M . 



Proof of Proposition 2.1. Let Kx,K 2 be compact subsets of M, then there exist Xi G Ki, such 
that f(Ki) — f(xi), for i — 1,2. By the definition (2^8), for a fixed 5 > 0, there exists e > 0, such 
that 

e < d H (K 1 ,K 2 ) + 8, 
Ki C KI 

K 2 c KI (2.11) 

Therefore there exist j/j G JCi i = 1,2, such that, 

d(xi,y 2 ) < dj{(Ki, K 2 ) + 8 7 
d(yi,x 2 ) < d H (K l7 K 2 )+S. 



(2.12) 



Hence, by the definition of x\, x 2 and the Lipschitz regularity of / it follows, 

f{K x )-f(K 2 ) < f(x 1 )-f(y 2 )<Lip(f)d(x 1 ,y 2 ) 

< Lip{f){d H {Kx,K 2 ) + 5), 

f{Ki) - f(K 2 ) > f( yi ) - f(x 2 ) > -Lip(f)d( yi , x 2 ) 

> -Lipif^dHiK^K^+S). 



(2.13) 



The previous inequalities, being <5 arbitrary, proves 

- f(K 2 )\ < Lip(f)d H (K u K 2 ). (2.14) 
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Therefore / is a Lipschitz continuous map over Comp(M), and Lip(f) < Lip(f). 
On the contrary, if x\,x 2 € X, then by ( [2.9] ), we have 

\f(xi) - f(x 2 )\ = \f({xi}) - /({x a })| < Lip(f)d( Xl ,x 2 ) (2.15) 

that proves Lip(f) < Lip(f). 



3 Main Notations and Definitions 

In this section we introduce the main notations which are used in this paper. 

1. Let / be some real valued, Lipschitz continuous function over the domain T> C IR m . We refer 
to the Lipschitz constant of / as to 

r- (t\ \f(x)-f(x')\ 

Lip(f) = sup ; - . (3.16) 

{x,x')EVxV \ x x \ 
x 7^ x 1 

2. We recall the definition of norm of an operator S : IR Pl — * H P2 

||5|| := max \Sx\. 
i£E Pl 
\x\ = l 

It follows : 

\\S\\<y/pm max \S hil \ =: ^pm ||S|U- (3.17) 
h = l,...,p 2 

I = 1, • •• ,Pi 

3. If c e TR J we say c < if and only if a < for every i = 1, . . . , j. 

4. Let d > j be two integers, and II be the set of all multi-indexes tt = (ii, . . . with 1 < i\ < 
%2 < ■ ■ ■ < ij < d. Then for every n 6 II and u £ H d , let denote the projection of u over 
the coordinates specified by 7r, i.e. u T = [ui 1} . . . , Uj -) G IR' 7 - 

For every differentiable function / : V X A — * MP, with V C H d and A C H™ 1 open sets, 
and for any tt £ II, let Jpf- denote the Jacobian matrix of / w.r.t. the coordinates of v € V 
specified by tt, i.e. 

df \ df h 



.. / - ., • h,l = l,...,j 

5. Given 7r in II, d > j, we define the map 

^ : IR'' x HV -> (3.18) 

where, for every u £ M d , v £ Hf , 

(^' r (u,u)) 7r = u, Z?(u,v) = Ui, Vigir, i = l,...,d. (3.19) 

In other words Z^iu^v) is a obtained from it, by substituting the vector v to the components 
of u corresponding to tt. Obviously if j = d, we have the only 7r = (1, 2, .., d) and Z^{u, v) = v. 
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6. For every it = (ii, . . . ,ij) € II, let denote by T v , the matrix 

{T ^ h ' 1 = { otherwise (3 - 20) 
for every 1 < h < d, and 1 < I < j. 

7. We call regular arc a function 7 : X — > H m , where X is a compact interval, which is piecewise 
differentiable on I, with 7' bounded and 7'(i) 7^ 0, for every i e I where the derivative exists. 



Definition 3.1 A non empty subset X C H m satisfies the property (CON), if X is connected and 
there exists a characteristic number a(X) > 0, such that for every pair of distinct points x±, xi £ X , 
there exists a regular arc 7 : [wi , W2] — > X such that the following conditions hold true: 

j(u>i) = Xi, i = 1, 2, 



1(7) < a{X)\ Xl -x 2 \, (3.21) 
where l(j) := f™^ \Y(t)\dt, denotes the length ofj. 

Obviously, if X is a convex subset of H m , then the property (CON) holds with a(X) = 1. Also, it 
can be proved that any connected compact submanifold of H m satisfies the property (CON) . 



Definition 3.2 A set Ai C H™ is called a d-dimensional Lipschitz manifold if, for every u £ M., 
there exists an open neighborhood W of u in M and an homeomorphism ip : W — > V, where V is an 
open subset ofJR d and such that ift and ip^ 1 are Lipschitz continuous. The couple (W,ip) is called a 
local chart for u. 



Definition 3.3 An atlas on a compact Lipschitz manifold M C 1R™ is a finite collection of charts 
{(W a , ip a )}a£A such that {W a } a £A is a covering of M. 



Definition 3.4 Let M. be a Lipschitz manifold, we define AflZ(M) to be the set of nonregular points 
of M, that is, for every point u S M\N~1Z(M.) there exists a local chart (W, tp) for u whose inverse 
is C 1 in a neighborhood ofip(u). Such a chart will be called a regular chart for u. 

Up to a suitable change of the atlas, in the reminder of the paper we make the following assumption: 
Assumption 3.5 // u 6 M\NTZ(M), every local chart (W,if>) for u is regular. 



Remark 3.6 With Assumption 3.1, the Definition 3.4 becomes 



M\MTZ(M) = {u G A4 : every local chart (W, ip) for u is a regular chart for u} 



Remark 3.7 Let M. C M™ be a Lipschitz manifold with dimension d. 



G 



1. We denote by T U (M) the tangent space to M. atuEAi. T U {M.) exists Hj^- almost everywhere 
on M. by Rademacher's Theorem, where Hf^ is the d- dimensional Hausdorff measure on A4. 
Since M C IR™, we can view T U (M) as a linear subspace of IR™. In particular, if u (ft NlZ(AA) 
and (j> : V C IR'' — ► <fr(V) C M., 4>{v) = u, is the inverse of a regular chart for u, the 
n-dimensional vectors 

form a basis ofT u (M). 

2. A function f : M. — ► IR- 7 is differ entiable at u NlZ{M) (resp. C 1 in a neighborhood of u) 
if for every chart ip defined in a neighborhood of u with an inverse differ entiable at ip(u) (resp. 
C 1 in a neighborhood of ip(u)), the function fo is differ entiable at ip(u) (resp. C 1 in a 
neighborhood of ip(u)) in the usual sense. 

3. Let / : M. — > TR? be differ entiable at u ^ JVlZ(M). The differential of / at u is a linear 
operator 

d u f : T U {M) - W (3.23) 

defined as follows. Let 4> be the inverse of a regular chart for u, as in 1., and let be given the 
basis (3.2^ ) of the tangent space T u (Ai). Then, for any w G T u (Ai), 



w = y^Wi——(v), Wi€JR,i = l,. 

^— 1 



The differential of f at u is defined as 

dufM = y,Wi^^{v) G IR J . (3.24) 

It can be proved that this definition is well-posed in that it does not depend on the choice of 
the chart <fi . 

Given the immersion of T U (M.) in IR", we can view the differential of a map as a linear op- 
erator over a d-dimensional linear subspace of IR™ and therefore we can consider its norm. In 
other words M. is a Riemannian manifold. 

The same definition of differential holds if f depends also on a variable x £ A C IR m . In 



this case definition ( 3.24 ) applies to f(-,x) for every x G A, and its differential is denoted by 
d u f(-,x). 



Definition 3.8 Let M. C IR™ be a Lipschitz manifold and f : M. X A — ► IR- 7 , A an open subset of 
IR m , X C A. We define NlZ(f) to be the set of nonregular points of f , i.e. 

AfTl(f) = {«£ M\JVTZ(M) : 3 x G A : f is non- differ entiable at (u,x)} . (3.25) 

J\fS(f,X) = {u G M\ (Af1l{M) UAfH(f)) : 3 x G X : f(u,x) < and d u f(-,x) is not surjective} 

(3.26) 

and 

V(M,X, f) = NK(M) U MTZ(f) U AfS(f, X) (3.27) 
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Definition 3.9 For every u £ M\NlZ{M) , we choose, once and for all, a regular chart for u, 
whose inverse will be denoted by 4> u , 4> u {v) — u. We define, for any tt £ II, the linear operator 

T" : M J — > T U {M) (3.28) 

whose associated matrix, w.r.t. the canonical basis ofJR- 1 and the basis 

d(j)u, ■. d(j) u 



ofT u (Ai), is the matrix T n defined in (3.2C) 



4 The Lipschitz Regularity of the Value Function 

In order to prove that the value function is Lipschitz continuous we need to prove that the map Uk{- 



defined in (1.4) is Lipschitz continuous with respect to the Hausdorff distance du in Comp{X^). 
We present some regularity results for multifunctions which have the same structure as the set of 
admissible controls Ql.4j ). 

Theorem 4.1 Let c = (ci, . . . , Cj), with a, i = 1, . . . ,j, j < n, real valued maps defined on M. x A, 
where M C H and A C TR m is open. Let X be a non empty subset of A which has the property 
(CON), and let A4 be a compact Lipschitz d-dimensional manifold, with j < d. We further assume 
that 

i) For every x € X the set 

U{x) = {ueM : c{u,x)<0} (4.29) 

is non empty. 

ii) T>(M,X, c) is closed and one of the following assumptions holds: 

(A) The set {(u,x) : x £ X, u £ U(x) n V(M, X, c)} = 0. 

(B) T>(M,X,c) 7^ and every u £ U{x) D T>(JA,X, c), with x £ X, is of adherence for 
U(x)\V(M,X,c). 

Hi) The function c := (ci, . . . , cj) is C{M x A) n C l {M\V(M, X, c) x A). 

iv) For every x £ X and u £ U(x)\T>(M, X, c), 

t := sup sup T(u,x) < oo . (4.30) 

x£X ue U(x)\V{M,X,c) 

with 

T(u,x) := max (cJ u c(-, x) o T%) 1 od x c(u,-) , (4-31) 

and 

II(u,x) := {n £ LT : d u c(-, x) o T% is invertible} . (4.32) 
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Then the map 



x £ X i ► U(x) 



(4.33) 



rs 



if m)-(A) holds : dn-Lipschitz continuous and its Lipschitz constant can be estimated by a(X)rLipM, 
where a(X) denotes the characteristic number of X as in Definition \3.f\ and 



LipM '■= sup {Lip((f>) : (f> 1 is a chart over A4} . 



(4.34) 



if u)-(B) holds : dn-uniformly continuous on every compact subset of X which has the property 
(CON). 

Next result strenghtens the regularity assumptions on the constraint function c in order to obtain 
Lipschitz regularity of the value function even in the case ii)-(JB). 



Theorem 4.2 In the hypotheses i),m)-(B), Hi) of Theorem 4-1, we assume that 

iv) there exist fi, X, r > such that for every x G X and u G U(x)\D(M. 1 X, c), and n € n(it, x), 



(d u c(;x)oT%Y 



< A, 



(4.35) 



\\d y c(u, -)|| < V y e A, \y - x\ < r 



and that 



v) if r is chosen as in iv), there exists L > such that for every u € U{x)\D(M., X, c) with x G X , 
and for every regular chart (W, ip) for u it holds: 



^(v,y)-^(v u ,x) 



<i(|«-«u| + \y-x\) 



(4.36) 



for every v in a suitable neighborhood of v u , y £ A, \y — x\ < r, with = c(ip 1 (-),-) 7 

v u = ip(u). 



Let 



t = A/i, 

then x G X *—>■ U(x) is dn-Lipschitz continuous, with constant a(X)rLipM- 



(4.37) 



Remark 4.3 // (A) holds true, then for every compact set K C X , we have 

{(u,x) : x G K, m G U{x) n V(M, X, c)} = 0, 
hence, by compactness, there exists a* > such that U(x) n (T>(A4, X, c)) a — 0, for any x G K. 



Using these results, we prove a regularity result for the value function fll.q ). Then wc prove Theorem 
O and Theorem fi~2 . 



Theorem 4.4 Let be given (1. 1)-(1. 4) and the related optimization algorithm (l.t). Suppose that: 
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1) for every k = 0, . . . , N — 1, the triplet (A4 k , c k , X k ) satisfies the assumptions of Theorem {.1 
under the condition m)-(A), or the hypotheses of Theorem {A. 



2) for every k = 0, . . . , N — 1, there exists a nonnegative, £ k -measurable and pk-integrable function 
y e^H V k (y), i.e.: 



V k (y)dp k (y) < oo, 

such that for p k — a.e. y G y k , 

\fk(x,u,y) - f k (x',u',y)\ < V k (y)\(x - x', u - u')\, 

V x,x' e X k , V u,u' e M k . 



(4.38) 



(4.39) 



If g is Lipschitz continuous over Xn then, for every k, J k is Lipschitz continuous over X k and the 
following estimate holds: 



Lip(J k ) < L(J fe+1 )lE fe [V r fe ](l + a k T k Lt PMk ), k = 0, ... ,N - 1 
Lip(J N ) = Lip(g). 



(4.40) 



with a k the characteristic number of X k , given in Definition 3A_, and r k defined in ( 4-3C ) or ( 4-31 ) 
according to the two alternatives of assumption 1). 



We prove this result first, then we prove Theorems 4.1 and 4.2 



Proof of Theorem |4.4| . We proceed by induction over k. For k — N, the function Jn is the utility 
function g, which is supposed Lipschitz continuous over Xn- We assume J k +\ Lipschitz continuous 
over X k+ i, for k < N — 1. Therefore, the function 

Vk(%,u) =JE{J k+1 (f k (x,u,y k ))] , (x,u) g X k x M k . 

is Lipschitz continuous over X k x A4 kl and an easy computation yields 

Li P (* k ) < Lip(J k+1 )JE k [V k ]. (4.41) 

Moreover, for any k = 0, . . . , N — 1, the assumption 1) implies that the set-valued map x G X k i— > 



U k (x) is dif-Lipschitz continuous, by Theorem 4.1 or Theorem 4.2, with Lipschitz constant estimated 



by a k r k Lip_\4 k . Hence we can apply Proposition |2.1| , to deduce that the marginal function defined 
on X k by 

x G X k i — ► max ^ k (x,u) 

uGU k (x) 



is Lipschitz continuous. The DP-algorithm (1.6), implies that this map is exactly J k . Moreover, by 



Proposition 2.1, its Lipschitz constant is estimated by Lip(ty k )(l + Lip(U k )). Therefore we have 

Lip(J k ) < Lip(J k+ i)JE k [V k ] (1 + a k r k Lip Mk ) , 
which proves the assertion. 
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In order to prove Theorem 4.1, we have to estimate the d_y-distance between U(x) and U(y) i.e. we 
need to prove, by the definition of du, an inclusion of the type U{x) C (U(y)) s , where S > 0. 
We need therefore to take an admissible control u for x and to show that we can "perturb" it to an 
admissible control for y whose distance from u is less than 6. The idea of the proof is to show that 
this property holds true locally and that the radius of the neighborhood where the property holds 
true is in dep endent of the point x and of the control u. This independence will allow us to prove 
Theorem hi by "iteration" , i.e. by covering the arc between x and y with a finite number of balls 
of constant radius where the property holds true. 



Lemma 4.5 In the same hypotheses of Theorem (.1 and according to the two alternatives of as- 
sumption ii ), we have: 



(A) for every regular arc 7 : [101, W2] 
which satisfy 



► X, there exists ro > such that, for every 101 < t < s < w% 
7([t,s])cBMt)), (4.42) 



the following inclusion holds: 

17( 7 (*)) C ([/(7(s))) T \ V t' > T t .sLip M , 

with 



T t,. 



rl(r,t,s) 



|7(£M, 



(4.43) 



(4.44) 



and t is given by (4-3C ). 



(B) For every compact set K C X which has the property (CON), and for any a > 0, there exists 
r (j G (0, a] such that, for every x,y € K such that 



we have 



\x -y\< TV, 



U(x) C ([/(2/)) r ' +CT , V t' > a{K) T Lip M \x - y\. 



(4.45) 



(4.46) 



with a(K) the characteristic constant for K, given in Definition 3.1 



Let's assume for the time being that Lemma 4.5 holds true. Using this result it is straightforward 
proving Theorem 4.1: 



Proof of Theorem 4.1. If X is a single point set, then we do not need to prove anything. Let's 
assume that X contains at least a pair of distinct points x\, xi\ we want to estimate the Hasudorff 
distance between the corresponding admissible control sets, i.e. 



d H (U(x 1 ),U(x 2 )) 



(4.47) 



By the propert y (CO N), there exists a regular arc 7 : [101,102] — ► X which connects x% to X2 and 
which satisfies ( 3.21 ). We have to distinguish between the two alternatives in assumption m)-(B): 

Case (A): For every t £ [w\, W2] let denote by B t the open ball centered in j(t) with radius ro > 0, 
obtained by applying Lemma 15, case (A), to the arc 7. We can assume that the curve does not 
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have self-intersections, otherwise we could define a new arc connecting the same points and having 
smaller length. We introduce the following sequence 



t := wi 



ti+i := sup{u; 2 > t > U : | 7 (t;) - 7 (s)| < a V s G fe, t]}, if * > 0, 



(4.48) 



that is increasing in the interval [iui, 102]. It is straightforward proving that t p — w 2 for some integer 
V > 0. 

Therefore {[toi £1] , . . . , [t p -i, t p ]} is a finite covering of [w±, w 2 ]. Moreover 7 ([ii, ti+i]) C B ti . 
By ([4.43D, we obtain 



C (C/( 7 (i l+1 ))) n , i = 0,...,p-l, 



(4.49) 



where r, = rLipM J t * ,+1 | 7 (0I^£ + J , e is an arbitrary positive number. By iterating the previous 
inclusion (4.49), we obtain 



u( Xl ) c {u{x 2 )) TLipMlM+e . 



(4.50) 



By switching the role of x\ and x 2 and by (3.21), the definition of the metric dn and the arbitrary 
choice of e, we can estimate (4.47) with a(X)rLipM\xi ~ x 2 \- 



Case ( B): Let K be a compact subset of X which satisfies the condition (CON). By ap plying 
Lemma |4~5| in the case (B), for any a > 0, there exists r„ £ (0, er], such that the inclusion (4.46) 
holds, whenever x,y € K satisfy (4.45). This implies 



d H (U(x), U{y)) < (1 + a(K)TLip M )a, V |z - y| < r CT , 
and therefore [/(•) is uniformly continuous over K, with respect to du- 



We turn now to the proof of Lemma L5. In the case (B), we need to approximate the controls which 
lie in a neighborhood of T>(A4, X, c). 



Lemma 4.6 Let M. be a manifold as in Theorem J^.l, K C X compact, and suppose that assump- 
tions i), m,)-(B) and Hi) hold true. 

For every a > 0, there exists p > such that for every x € K and u £ U (x), there exists u a £ U(x), 
such that 

d(u CT ,V(M,X,c))> p, and \u a - u\ < a. (4-51) 
Proof. We prove the result for the controls u £ U{x) fl (T>(Ai, X, c)) p , otherwise it suffices choosing 

U a = U. 

Suppose by contradiction that there exists a > such that for p = Xi there exist Uh £ U(xh), 
Xh £ K with 



d(u h ,V(M,X,c)) < 



(4.52) 



such that, for any u £ U(xf l ), 



d(u,V(M,X,c)) < —, or \u-u h \>a. 



(4.53) 
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Without loss of generality we may assume that Xh — > 3;, it/i — * w, as h — > oo, for some x G if and 
m € T>(Ai, X, c), since if is compact and T>(A4, X, c) is closed by assumption ii) of Theorem 4.1. 
The continuity of c over M x i, implies u G U(x) (1 T>(Ai,X, c) and, by m), u is of adherence for 
U(x)\T>(M, X, c), therefore there exists m G J7(x)\Z>(.M, X, c) such that 



\u-u\< -. 



(4.54) 



Let 7r G x) and 4> — 4>u be the map, relative to u, fixed in Definition 3.9, with </>(«) = w, u G IR^. 
Let Z^ be the function introduced in Section |[ and 



F(w,y) := c(<j>(Z*(y,w)),y) -c(u,x) G H J , V G H J x A, 

which is of class C 1 in a neighborhood of ((-D) 7r ,x). We observe that 

F((v)„,x) = c(<f>(v),x) - c(u,x) = 

and 

dF dc(<j>(-),x) , dZ* 



(4.55) 
(4.56) 

(4.57) 



where the first matrix on the right-hand side is j x d and the second one is d X j. It is easy to verify 
that if 7r = (ii, . . . , ij), 1 < «i < %2 < • • • < ij < d, and if 1 < z < d, 1 < / < j, we have 



dZ 7 * 
dw 



1 i = ii 
otherwise 



(4.58) 



By the definition of tt we deduce the invertibility of the Jacobian matrix (4.57). With (4.56) this 
allows for the application of the classical Implicit Function Theorem which implies the existence of 
a continuous map q : B — > IR-', where B C A is an open neighborhood of x in JR m , such that 



q(x) = (v)„ 



and 



F(q(y),y)=0, VzjGB. 
Since c(u,x) < 0, using (|4~55| ) and flOjCj), we get 



(4.59) 
(4.60) 
(4.61) 



For large values of h, we have x^ G B and since q(xh) ~ * u and uj, -> n as /i -* oo, by (4.54), we 
also may assume 

\q(xh) - u h \ < a. 



Hence by (4.61) and (4.53) with u — q(xh), we infer 

d{q(x h ),V(M,X,c)) < ~, 
h 



(4.62) 



for large h. Letting h — > oo in ( 4.62 ), we obtain 

u = q(x) G V(M,X,c), 

which is a contradiction. 
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The main difficulty in the proof of Lemma 4.5 is in building the radius r$ independent of t, s and of 
the controls in U(^(t)). We use a quantitative version of the classical Implicit Function Theorem, 
which provides an estimate of the radius of the balls where the implicit map is defined, see j7j. 
Using the implicit function theorem, we are able to build a map that, for every state 7(5) "near" 
j(t), prescribes how to modify the control u G U("f(t)) to obtain an admissible control for y(s). 

Theorem 4.7 Let F : O — * H- 7 be a map defined in the open set 0cE 3 X H m . Let r\, r%> be 
such that, if Bi = {v G W : \v - v \ < n} and B 2 = {y G M m : \y - y \ < r 2 }, then B x x B 2 C O 
and the following hypotheses hold: 

dF 

F, — — are continuous on B% x B 2 , (4.63) 

dv 

dF 

F(v o ,y Q ) = 0, det— (v ,y ) ^ 0, (4.64) 



sup \F(v Q ,y)\ < 2jp^||, T = (If (v ,yo)) 

yEB 2 

sup H^-Tof || < I, 

B 1 xB 2 



(4.65) 



where Lj is the identity matrix of order j . Then there exists a unique function q G C(B 2 ; B\) which 
satisfies 

q(yo)=v , (4.66) 

and for every (y,y) G Bi x B 2 , it holds 

F(v,y)=Q v = q(y). (4.67) 

Next Proposition provides an expression for the Jacobian matrix of the implicit function q. 

Proposition 4.8 Let D\, D 2 be two open balls of & and H m , respectively. Suppose that F G 
C 1 (Z?i x D 2 ;W(?), and q G C(D 2 ;D\) satisfies F(q(y),y) — 0, for every y G D 2 . If ^ is invertible 
in D\ x D 2 , then q G C ll (Z?2) and 

^(y) = -(^(g(y),y)) for y g d 2 . (4.68) 



We turn finally to the proof of Lemma 4.5 



Proof of Lemma 4.5. We prove the assertions in steps. We derive an approximation of U(x) by 
the controls of U(y) for x,y which lie in a compact connected subset K of X, then we specialize the 
discussion according to the assumptions ii)-(A) and m)-(B). 

(Construction of a covering). Let K be a nonempty compact connected subset of X, a > such 
that 

K w c A. (4.69) 
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Let a* be chosen as in Remark 5.8 and a < a. We introduce 

a* if (A) holds 



P 



p a if (B) holds, 



where p G is related to a and to the compact set K C X via Lemma 4.6. Let 



K~, 



{(u,x) : x € K, u e U(x) and d(u,V(M, X, c)) > p}. 



(4.70) 



(4.71) 



By Remark 4.3, in the case (A), and Lemma 4.6, in the case (B), for small tr, this set is a non empty, 
compact subset of A4 x K. For every (u,x) S A' 7iP , let tt(u,x) S II(u,a;) and 0„ the map given in 
Definition 3.E, for the point u, <fi u : V u — > M.. We may assume, without loss of generality, that 
4> S C 1 (V U ), furthermore, since T>(A4,X, c) is closed, we may also suppose (j) u (V u ) nl)(A4, X, c) = 0. 
By the assumption Hi), the map ip u , defined by 

ip u (v,y) = c(cj) u (v),y), V (v,y) G V u x A, 

is C . Furthermore, the matrix 



R(w,y;u,x) 



dv 



is invertible at (u, x). 

By the continuity of the function ( 4.72] ), there exists 5' = S'(u, x) > such that 

z(u,x) := irrf {|det R(w,y;u,x)\ : (w, y) G {B s ,(u) n M) x B s ,(x)} > 0, 



(4.72) 



(4.73) 



where B$i{u) C H™ is the open ball of radius <5' centered at u, with Bs>(u) flM C <t>u{Vu)t an d 
Bs'(x) C -£T cr / 2 is the open ball in lR m of radius 5' centered at x. By the continuity of there 
exists S(u,x) < 5'(u,x) such that 



l-fcV) - <fc V)l < y , v w g B tf(lVB) («) n M. 



(4.74) 



with > such that the closed ball in H d centered at 0^ 1 (u) with radius ( u is contained in V u . 
Such a ball is denoted by N u . 
The collection 

^Bi(u) x B±(x) : 5 = 5(u,x), (u, x) G i^ 7 . p | 

is an open covering of K^^ p , therefore we can extract a finite covering corresponding to some points 
(til, . . . , (itp, Xp) G K ltP . To simplify the remainder of the proof let us define: 

(f>i ■■= <t>v,i, ¥>i-=<Pui, Ri(; •) := R{; ■;u i ,x l ), T 4 := T£\ Lj := Lip{cf>i), 

7r 4 := n(Ui,Xi), z t := z(u tl Xi), 5{ := S(ui,Xi), Ci '•= Cm 

tfi—ATui, O i (S):=(Bs(u i )nM)xB s (x i ), V6 > 0, i = l,...,p, 



and 



min i5i, z* := min z^ 

l<Z<p l^;*<p 



(9* 



min ^r, £* := min (i 



i<i< P J 



l<i<p 



By (4.73), we have 



|det Ri (w,y) | > z* > 0, V £ O^,), V? 



(4.75) 
(4.76) 



15 



Now suppose (u, x) € ^7,p, then (u,x) <E 0,:(4f), for some i = 1, . . . ,p. Let 



and 

given by 



F : HV ' x A — ► HV 

F(i;,y) := ^ (Z^^v^v)^) - c(u,x) 



(4.77) 



which is defined and C 1 in the open ball in TR d centered at and with radius y, since (4.74) 

implies 

1^^,^)- 0^(^)1 <Q 

that implies Z' Ki {vi,v) € T4 



In order to apply Theorem 4.7 to _F with (v p, yp ) — {{Vi) Wi , x) , let us assume for the time being that 
fi,r"2 are chosen as prescribed by Theorem 4.7. We observe that F(vo,yo) = and 



dF 

— ((vi) n ,x) = Ri(u,x), 



is invertible by ( 4.76| ). By Theorem 4.7, there exists a continuous map 

q ■ B 2 — ► Bi 



with B\ x £>2 C 1R-' x A, such that ( 4.66| ) and (4.67) hold true. The function q is a feedback function 
that allows us to build an admissible control for y starting from the admissible control u for x : as 



in the proof of Lemma |4.6| (see (4.61)), we have 

q(y) := & (Z^( Vl ,q(y))) e U(y)\V(M, X,c), 

for every y G B%. 



(4.78) 



(Construction of r\ and r-i). In order to construct r\,r% as in Theorem 4.7, we consider 



A := max 

i<i<p 



sup 

(w,s/)eOi(i) 



(i*Ktf)) _1 



(4.79) 



which is finite since for every / 



(Ri(w,y)) 1 



< 



Const 



and Const is a constant that depends only on the supremum of the norm of -^j- over a compact 
subset of H rf x TR m . The regularity assumption Hi) allows us to define 



Tx {u ' x) 



/i > sup 

u)(h,k) :— max sup < 
i<i< P I 

v, v'eM, y, y'e K^ 2 Y h, k > 



'EM, d(u',T>(M,X,c)) > p, i' e if ff/2 | (4.80) 
: \v-v'\<h, \y-y'\<k, 

(4.81) 



_(«,„)-_(„,„) 
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Let us fix 



r\ < mm 



in ( 



s* 9* 



C 

2 ' 2/3' 2/3' 4 



(4.82) 



Notice that the modulus u> defined in (4.81) is a descreasing function of h and k and its limit for 

1 



h., k — > is zero, so we may choose ri so that 

u>(rx,(3ri) < 



(4.83) 



Let us verify the inequalities in (4.65) with (vo,yo) = ((vi) ni ,x): 



sup \F(v Q , y)\ = sup \c(u, y) - c(u, x)\ < \ir 2 = < 777^-^, T = {Ri(u, x)) 1 (4.84) 

y£B 2 y£B 2 z * Z \\ 1 0\\ 

where we used d(u,T>(M.,X,c)) > p and x + ri(y — x) 6 K a / 2 , for every y € B2 and < rj < 1. 
Furthermore for any (1;, y) £ B\ x £2 we have 



<9.F 

/-To — («,») 



< llTn 



dipt, dipi 
-£-{yi,x) - — [Z *(vi,v),y) 
ov ov 

1 



I Til 



(4.85) 



This inequality follows by the definition of u> and by ( 4.83 ): in fact Vi, Z Vi {v^ v) € Nl, and r 2 < § 

implies x, y S K a l' 2 . This proves (4.65) and justifies the application of Theorem 4.7. 

The radius r 2 depends only on K and er, and it does not depend on the particular choice of (u,x). 

(Approximation of the control u). Again we need to distinguish between the alternatives (A) 
and (B) in ii): 



Case (A). Let K be the image of the arc 7 and ro = r^iK, a*). We consider s > t as in ( 4.42 ), 
where x — 7(f), and we have to approximate u by an admissible control for the state 7(s). To this 
purpose we apply Proposition 4.8 to the pair F, q obtained in the previous step. With the choice 
(E8§) for r l5 let 



D x := {welPJ : \v-(vi) m \ < 2r x } 
Z?2 is the interior of i?2- 

For any v £ Di and y £ D 2 , 

\<t>i{Z^{v h v))-Ui\ < ^(^(^.^-^(wiJI + lu-Uil < 2L,n + I < 5 4 , 

|y - Xi| < r 2 + \x - Xi\ < ^ + §■ < 

This implies 

«)),y) £ 0,0*0, V e Di x £» 2 . 

Therefore by ( 4.76| ), the matrix 

OF , 



(4.86) 



(«,y)=i? i (^(^(« i ,u)) I y) 



(4.87) 



(4.88) 
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is invertible over D\ x D 2 , and q(D 2 ) C Si C D\. By applying Proposition 4.8 we deduce that 
q e C 1 (D 2 ). Since 7 ([i, a]) C D 2 , for any f G [s, t], by (pL7^) 



The assumption w) implies 
9g 



9y 



(7(0) 



q(l(Z))eU( 7 (t))\V(M,X,c) 



[^(9(7(0).7(0)] _1 ^(9(7(0),7(0) 



d« 7 tt))c(-, 7 (0) o d 7(?) c(g( 7 (e), 



< T(g( 7 (0),7(0)< 



(4.89) 



(4.90) 



In fact the matrix associated to the linear operator g^( 7 (£)\c(-, 7(C)) ^wf 7 5 with respect to the 
canonical basis of H J , is Ri{q{"l {£,)),! {£,))■ Using (4.90) we obtain 



|$(7(«))-«| < L<PA<|«(7(a))-?(7(t))l 

< rLip^ ^ KOOK < T t,sLipM . 

This inequality proves that 

u e (U('j(s))) T , V r > Lip M T ttS . 
Since r2 is independent of (it, a;), we have 

E%(t))\ (^,i,c))rc([/( 7 ( S ))r'. 



(4.91) 



(4.92) 



(4.93) 



Case (B). Let if be a compact subset of X, which has the property (CON), and r a = min(<7, r2 ^(K)^ )• 
Let x,y € if be such that |x — y| < r CT . Then we consider a regular arc 7 : [101,102] —> if as in the 
Definition 3.1, which connects x to y and lies in if. For every £ € [101,102], we have 



17(0 - 7K)I < *(7) < ~x 2 \< ra, 



which implies 



l([wi,W 2 ]) C 5^(7(101)). 
Hence by the same arguments developed in the previous case, we deduce 

U(x)\(V(M,X,c))) p C (U{y)f , V t' > a(K)TLi PM \ Xl - x 2 \ 

(Conclusions). 



(4.94) 



(A) Using the inclusion ( 4.92 ) and the definition of p in ( 4.7C ), we infer 

[%(*)) c (t/( 7 ( S ))r . 



(4.95) 



In fact, in this case, p = a*, which is defined in Remark 5.8 so that U(x) C (T>(M, X, c)) c 
for every x £ X . 
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(B) Let x,y € K, u € U(x), with T>(A4, X, c)) < p = p a . By Lemma Q we find u a £ U(x), 
with X, c)) > p, such that 



U — Urr\ < a. 



Hence, using (4.94), there exists v a € U(y) such that |u — w^l < t' + ct, t' > a(K)Lipj^T. This 
implies 



[/(*) c (1%)) 



This concludes the proof of Lemma 4.5 



(4.96) 



Proof of Theorem 4.2. Let, for any u € M and for any local chart (W,ip) for u, 

5{u,W,ip) = sup{<5 >0: -B 5 (^(u)) c ^>(TV)} 



and 



<5(m) = max <5(w, TV, -0). 

(W,V>): tiSW 



(4.97) 



(4.98) 



Since every local chart tp is Lipschitz continuous and M. is compact in Hi", 6(u) is finite for every 
u £ M. We prove that the map (5(-) is uniformly bounded from below. Since M is compact, it 
suffices proving that £(•) is lower semi-continuous, i.e. that the set 



{u e M : 8{u) > a} 
is open in M, for any a > 0. Suppose, for the time being, that the set 

{u e TV : 6(u,W,ip) > a} 



(4.99) 



(4.100) 



is open in M. for any local chart (TV, ip) in the atlas of M. and for any a > 0. Then if u is such that 
S(u) > a, there exists a local chart (W,ip) for u such that 6(u, W,ip) > a. Therefore, there exists 
rj > such that S(w, W,tp) > a for any w S B v (u) n IV. Hence <5(w) > a for any «; G B v (u) fl TV. 
Since ll 7 is open in Ai, B v (u) n TV is open in A4\ and therefore <$(■) is lower semi-continous over M.. 
To prove that the set (4.100) is open in M, let u S TV be such that 6(u, TV, ip) > a and £ e 

(°- ^LW)"^ and 77 € ( "' W > ^ ~ Li P^0, then 



B n (ip(w)) C V(W) Vi»£Bj(u)nff. 



This implies 



(5(w, TV, -0) > rj > a V iu e S 5 (m) n TV. 
Therefore, since TV is open in VW, the assertion is proved. 



Let 



6 = min <5(?i) > 



(4.101) 



and r be the image of a regular arc 7 : [w±, 102] — ► X , which connects two distinct points i^ijEl 
and such that the property (CON) holds. Let W > be such that 



L ff C A. 
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Let x £T and u £ U(x)\T>(A4, X, c), it £ H(u, x) and 4> u : V — ► M, the map fixed for u in Definition 
|3.9| . We may assume, up to modifying from the beginning the choice of the local charts {(j> u }ueM 
that appear in Definition 3.9, that cj> u is such that 

%(<fcV)) c V. 

Let 4> = 4> u , W = 4>(V) and F the map 

F{w,y) ■.= (p(Z*(v u ,w),y)-c(u ) x)£M j , V (w,y) : Z* {v u ,w) £ V,y £ A, (4.102) 
with <p(-, ■) := c(<p(-), ■) G C 1 ( V x A) and = u, then 



(4.103) 



Since J r ((f M ) 7r , = and using ( 4.103 ), the assumptions ( 4.64 ) are sa tisfied. Therefore we can 
apply Theorem 4.7, with (vo,yo) := ((v u ) w ,x). The inequalities in (4.65) can be proved as follows, 
by assumptions iv) and v): let r\ and r 2 be given by 



r 2 = /3n, 



4 r l < mill (^3'^'^ 



4L(l+/3)v/djA 



(4.104) 



then since the matrix associated to the linear operator d u c(-,x) o T", w.r.t. the canonical base of 
TR j , is 



dip 



we obtain 



sup |F(u , y)| = sup \c(u, y) - c{u, x)\ < 

y£B 2 y£B 2 

sup ||d„c(u, -)||r 2 < /xr 2 = ~y < ^77, ■ 

2/GB2 2A 2 Po|| 



(4.105) 



As to the second inequality in ( 4.65| ), since 

\Z n {v u ,w) — v u \ < n V w £ Bi 

and 



\y — x\ < r 2 < r V y € B 2 



we have 



5F. 



ll/i-To^ (w,y)\\ <xV(dj) 



< \y/djL{ ri + r 2 ) < i. 



Moreover, by Prop osition 4.8, the implicit function q : D 2 — > £>i is C 1 (Z?2) ) where Z?! and Z?2 are 
define d as in ( 4.86 ). Let t > s, suc h that x = j(t), and j([t, s]) C D 2 , we derive the inequality 
(4.91), as in the proof of Lemma 4.5. Since r 2 and r± do not depend on u, we deduce the inclusion 

U(rf(t))XD(M,T,c) c (C/( 7 (s))) r ' , Vr' > T t ,.Up M < 
Taking the closure, by assumption (B), we conclude that 

E%(i)) C (£/(7(s))) T ' , V r' > r t , s iipM- 
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In the case (A), we can repeat the argument used in the proof of Theorem 4.1 substituting r 2 for 
ro. This brings to the inequality (4.50) which, as before, implies 

d H (U(xi),U(x 2 )) < a(X)rLip M \xi - x 2 \. 

This proves the Lipschitz regularity of £/(•)■ 



5 An Application to a Financial Problem 

We discuss here an application of the results in the previous sections to an optimal asset-allocation 
problem. More precisely we consider an optimal asset-liability management model in presence of 
constraints: the company can manage the investment coming from the policy-holders' payments, in 
order to satisfy several regulatory and solvency constraints and to achieve a given objective. 
The company can decide, at each time step, how to distribute the total wealth between the available 
assets to achieve its goal, but he has to obey a number of constraints. Furthermore, each portfolio 
adjustment entails transaction costs, since it means selling part of an asset to provide either liquidity 



or a different asset (see 17 , |18| for a detailed description of the financial model). 



We assume, for the sake of simplicity, that the manager can choose at each time step to, ■ ■ . , £jv— i, 
between a riskless and a risky investment, denoted by B and S, respectively, though the procedure 
described applies also to the more general case of n possible investments characterized by different 
values of yield and volatility. 

We assume that the transaction costs of moving wealth between the sections are paid only on 
buying and not on selling and that these transaction costs are linearly proportional to the size of 
the transaction. The evolution equations for the amounts invested in stocks and bonds are 

S k+1 = [S k (l - u fc ) + B k v k (l - X s )]Y k s , So = s Q > 

(5.106) 

B k+l = [B k {\ - v k ) + S k u k (l - X b )}Y b , B = b > 0, 

where for every < k < N — 1, (u k ,v k ) € M k C [0, 1] x [0, 1] represent the percentage of risky in- 
vestment that is moved to riskless investment and viceversa, at time t k = kAt, Y k s ,Y k b : O — s- (0, oo) 
are random variables and < X b , X s < 1 represent the transaction cost coefficients. 



The model is self-financed in that it does not require the use of cash to perform the transactions. 
After each portfolio adjustment, the investments evolve according to the stochastic yields Y^,Y k . 
We assume that the joint process {(Y k s ,Y k )} k is a discrete Markov chain which takes values in a 
sequence of finite discrete subspaces of (0, oo) 2 , {y k } k . The chain is characterized by the density 
function: 

p k (y s ,y b ) = P(Y k s =y s ,Y k b = y b ), V(y s ,y b )ey k . (5.107) 

The company has to satisfy at each time-step a regulatory constraint that imposes a limit on the 
percentage of wealth invested in risky assets. More precisely, the adjustment must be such that the 
fund after the adjustment satisfies: 

m-v k ) + s k u k (i-x») > a 



B k (l ~ v k ) + S k u k (l - \ b ) + S k (l - u k ) + B k v k {l - X s ) 



21 



i.e. the percentage of wealth invested in riskless assets, after the portfolio adjustment, is bigger than 
a. Furthermore, since y k is finite, we may require that the adjustment is such that the constraint 
is satisfied also at time ifc+i, 

Bk+1 > a (5.109) 



B k +i + S k 



+ 1 



with (Sk+ijUk+i) given by (|5.106p . 

It can be shown that this constraint is equivalent to the following inequality: 



c k (u k ,v k , S k ,B k ) := S k (l - u k ) + v k B k {l - X s ) - q k [B k (l - v k ) + u k S k (l - X b )} < 0. (5.110) 



with 



l-a . ( . y b 
qk — mm 1, mm — 

a \ (y°,y b )ey k y s 



We would like to avoid that any of the two in vestments becomes null at some time. To this aim, we 
observe that, because of the structure ( 5.106 ), if the initial allocation (so, bo) is such that (so, bo) € 
Xo, with 

X = {(S, B) : S, B > A , S + B < D }, (5.111) 
with Dq > Ao > it is possible to define A k , D k and S k such that if the control space at time t k is 

M k = [0,5 k ]x[0,6 k ], (5.112) 

then (S k ,B k ) € Xk with 

X k := {(S, B) : S,B>A k ,S + B< D k } (5.113) 
and the admissible control space at time t k , 

U k (S, B) {(u, v)eM k : c k (u, v, S, B) < 0} (5.114) 
is nonempty for a ny (S, B) G Xk- 

By the structure ( 5.110 ) of the constraints, if U k (S,B) ^ 0, then the control (5,0) must belong to 
U k (S, B) for some S £ (0, 1). Therefore, let 6 k € (0, 1) be such that 



c k (6 k ,0,S,B) = S(l-5 k )~q k {B + 5 k S(l-\ b )} < 0, V S, B > 0, 



and therefore 



S k = sup 



S-q k B 



S,B>oS(q k (l-\ b ) + l) qk (l-\b) + l 



6(0,1), V/c>0. 



(5.115) 



Assuming (Sk,Bk) € Xk we can recursively compute A^+i and Dk+\ that define Xk+i- Since 
Sk, Bk > Afe > and u,v £ M.k, we have 

S k +i > A fc (l - S k )y k 
B k+l > A k (l-S k )y.. 



where 

Therefore we define 



y k := min{min(y s ,y & ) : (y s ,y b )&y k ) 



A fc+ i := A fe (l - S k ) mm(y s k ,y b k ) 
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To obtain D k+ i as a function of D k , we observe that by equations ( 5.106| ), if S k + B k < D kl for 
every possible choice of the controls in A4 k , we have 



where 
that is 



S k+1 + B k+1 < D k y k , 

y k := max {max(y s , y b ) : (y s ,y b ) G y k ) 
D k+1 =D k y k , Vfc>0. 



We have proved that if (so, bo) £ Xp, wit h Xq as in ( pMll| ), equations ( |5. 106|) map X k x A4 k x y k 
into Xk+i, being X k defined as in ( 5.113 ). Furthermore the admissible control set defined in ( |5. 114 ) 
is non empty for every (S, B) € X k . 

As in section [j], we want to find an optimal investment strategy which maximizes the expected value 
of a given utility function g at time tpj- 



Since M. k — [0,5k] x [®,S k ] is not a Lipschitz manifold in the sense specified by Definition 3.2 



we 



decompose it as the union of infinitely many subsets, which are 1-dimensional Lipschitz manifolds. 
We can apply Theorem 4.2 to these manifolds and then extend the result to A4 k . 



Proposition 5.1 The map (5.114) is dn -Lipschitz continuous over X k , for every k>0. Moreover, 
if g : [0, +oo) x [0, +oo) — > IR is Lipschitz continuous, the value function J k (S, B), obtained via the 
DP algorithm \l.f\ ) is Lipschitz continuous over X k , for any k = 0, . . . ,N. 

Proof. We consider, for every < S < 5 k , the rectangle Rs :k = [6, 5 k ] x [0, dk], and we define 

M s ,k = dRs, k , (5.116) 

and 

U 5ik (S, B) := U k (S, B) n M s , k (5.117) 



for every (S,B) £ X k . We prove, by applying Theorem i/2, that the multifunctions ( 5.117 ) are 
Lipschitz continuous over X k and that their Lipschitz constant does not depend on the parameter 
5. 

Since the space X k is convex, a k — a(X k ) = 1; furthermore Ms,k is obviously a Lipschitz manifold 
in H 2 of dimension d = 1 = j . 



We want to apply Theorem 4.2 to derive the Lipschitz regularity of Us,k- The assumption i) in 
Theorem 4.1 is a consequence of the fact that Ug, k {S,B) contains the control (S k ,0), for every 
(S, B) £ X k and 8. Now let us observe that 



V(M s , k , X k ,c k ) = {(5, 0), (S k , 0), (S k , S k ), {6, S k )}. 
Let (S, B) £ X k and (u, v) £ M.s,k\NTZ(M.s,k), the inverse of a local chart for (u, v) is 



(5.118) 



(t,0) 



8<t<Sk,ifv = Q 



(S k ,t) < t < S k , if u = S k 
(t,S h ) 5 <t<5 k ,iiv = 5 k 
(5,t) 0<t<5 k , ifu = S. 



(5.119) 
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The constraint function c k is regular (AflZ(c k ) = 0) and the jacobian matrix at (it, v) of the restriction 
of Cfc to the manifold Msm is, by ( |5.110D , 



Jc k (u,v) = 



-S[l + q k (l- A 6 )] ifv = 0orv = 6 k 
B(l — X s + q k ) if u = 6 or u = 5 k . 



(5.120) 



Therefore II(it, v, S, B) is non empty. This proves the assertion (5.118). 

Now, V(Ms, k ,X k ,c k ) is closed and, if (u, v) G T>(Ms,k, X k , Ck)\{(Sk, 0)} is an admissible control for 
(S, B), it can be approximated by 

( (5 + e,0) if (u,v) = (5,0) 
(u £ ,v e ) — < (6k,S k -s) if (u,v) = (5k, Sk) 
[ (S + e,S k ) if (u,v) = (6,5 k ) 

for < e < 5k, 5k — 5. In fact (u e , v e ) € Us y k(S, B)\V(Ms,k, Xk,Ck), as follows by the monotonicity 
of the constraint function Cfc, w.r.t. u and w. If u = 5k and w = 0, then for every (S,B) £ Xk, by 
Q5.115D it holds 

c k (u,v,S,B) = ~q k B < ~q k A k < 0, 

therefore, by continuity, we can approximate (5 k , 0) with admissible controls of Ug k (S, B)\D(M.g k , X k , c k ). 
In other words the assumption m)-(B) of Theorem 4.1 holds true. 



To prove that the assumptions iv),v) of Theorem 4.2 hold true, we observe that 
|| G [0, 1], || S \-q k , -q k + 5 k (q k + 1 - X s )}. 

Therefore, the assumption iv) follows by choosing 

5k 1 



X k := min — 



A fc ' A fc (l-A s 



qkj 



fJ-k ■= Vl+ [max(g fc , \5 k (q k + 1 - A s ) - q k \ 



(5.121) 

(5.122) 
(5.123) 



and for any r > 0. 

The assumption v), follows by the linear dependence on the st ate of the system of ( 5.120| ). We can 
construct an atlas over Ms,k by taking the parametrization (5.119) for the points which are not 
vertices and by defining compatible charts on the vertices so that 

Lip Ms . k < 1. 



Therefore, Theorem 4.2 implies that U$^k(', •) is Lipschitz continuous over X k and its Lipschitz 
constant is estimated by X k [i k - Now observe that 



U k (S,B) = \JU s , k (S,B), 



for every (S, B) £ X k . Therefore by the definition of the metric dn, we have 

U s , k (S,B) c (U s , k (S',B')f c (U k (S',B')f , 

which yields 

U k (S,B) c (U k (S',B')f, 
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for every (5, B), (S',B') £ X k and a > \k^k\{S — S',B — B')\. This implies the Lipschitz regularity 
ofU k (-,-). 

To prove the Lipschitz regularity of J)., it suffices proving that assumption 2) in Theorem 4.4 holds 
true. In fact we can repeat the proof of Theorem 4.4 by replacing assumption I) by the Lipschitz 
regu larity of Uk{-, •)■ 
By ( 5.106|) , we have 



f k (S, B, u, v, y s ,y b ) = ([S(l - u) + Bv(l - X s )}y s , [B(l - v) + Su(l - \ b )]y») , 



(5.124) 



for each (S, B,u,v,y s ,y b ) G Xk x Mk x 34- Using the linearity and ( |5.113 ), it can be proved that 



/*(• 



y s ,y ) is Lipschitz continuous over Xk x Mk and its Lipschitz constant is given by 



yll + D 2 k + {l- min(A s , \ b ) f max(y s , y"), 



which is pfe-integrable. 
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